Multimodal search

ABSTRACT

A trained processor is described which operates beyond an untrained point. Information is stored in a memory array in a tree-allocated file. Information is stored in the memory as key functions with associated trained responses. After the processor has been trained, it is able during an execution cycle, to find appropriate responses for query key functions. These query key functions are compared with the reference key functions stored in the memory array to find an appropriate trained response. During the execution cycle, there are some query keys for which there is no corresponding reference key function stored in the memory array, therefore, no readily apparent appropriate trained response for the query key. These query key functions for which there is no readily apparent trained response are termed untrained points. Thereupon a query key function which constitutes an untrained point is effectively compared with the reference key functions stored in the memory array to establish and store a difference function for stored reference key functions. Logic means then selects for the untrained point a trained response from those trained responses best satisfying a predetermined decision criteria. During the comparison operation, conditions are measured that indicate when reference key functions corresponding to a given group of trained responses cannot be an appropriate response for the untrained point in question. Logic means waive further examination of some stored reference key functions thereby greatly expediting the efficiency of search. In the multimodal search there are several modes of search, each mode further defining the item being searched.

United States Patent Masten et al.

[54l MULTIMODAL SEARCH [75] Inventors: Michael K. Masten, Richardson; William C. Choate, Dallas, both of Tex.

[73] Assignee: Texas Instruments Incorporated,

Dallas,Tex.

[22] Filed: June 1, i970 [2i] Appl. No.: 42,429

[52] U.S.Cl ..340/l72.5 [51] Int. Cl. ..G06f 15/18 [58] Field of Search ..340/I72.5, l46.3

[56] References Cited UNITED STATES PATENTS 3,388,381 6/l968 Prywes et al. ..340/l72.5 324L124 3/l966 Newhouse ...340/l72.5 3,346,844 l0/l967 Scott et al. ...340/l72.5 3,551,895 l2/l970 Driscoll,.lr ...340/l72.5 R26,772 l/l970 Lazarus 340/1725 3.333349 7/l967 Clapper ..340ll72.5 3,309,674 3/[967 Lemay ..340/172.5 $074,050 [H963 Shultz ..340/l72.5

ABSTRACT A trained processor is described which operates beyond an untrained point. Information is stored in a memory array in a tree-allocated file. information is stored in the memory as key functions with associated trained responses. After the processor has been trained, it is able during an execution cycle, to find appropriate responses for query key functions. These query key functions are compared with the reference key functions stored in the memory array to find an appropriate trained response. During the execution cycle, there are some query keys for which there is no corresponding reference key function stored in the memory array, therefore, no readily apparent appropriate trained response for the query key. These query key functions for which there is no readily apparent trained response are termed untrained points. Thereupon a query key function which constitutes an untrained point is effectively compared with the reference key functions stored in the memory array to establish and store a difference function for stored reference key functions. Logic means then selects for the untrained point a trained response from those trained responses best satisfying a predetermined decision criteria. During the comparison operation. conditions are measured that indicate when reference key functions corresponding to a given group of trained responses cannot be an appropriate response for the untrained point in question. Logic means waive further examination of some stored reference key functions thereby greatly expediting the efficiency of search. In the multimodal search there are several modes of search, each mode further defining the item being searched.

4 Claims, 32 Drawing Figures GO TO ENTRY NODE 0F TREE CALCULATE ME AT soro YES IS THERE AN no FALTERNATE ALTERNATE NODE none TSAME LEVEL Y ARE WE N0 g g a ES FINISHED WITHTREE 6/ THIS NODE 53 N0 DO VIE ACCEPT NUDE ARE WE AT LEAF LEVEL GO TO LINKED NODE AT NEXT LEVEL GO TO NEXT AVAILABLE NODE AT PREVIOUS LEVELS 56 ADD/OR \r REPLACE RESPONSE PATENTEDFEMSIBTS 3716.840 SHEET 03 0F 20 J '23 l 3 1 MEMORY UPDATE 4 II |36\ I32 QUANTIZER Q5 STORAGE x 5 (G MATRIX) 3o QUANTIZER i i l L v-MEMORY is L UPDATE QUANTlZER STORAGE 136 u (A MATRIX) 26 um SOURCE F- I --J 3200s I 32004 I 32003 I 32002 J I seem rl -4 -3 -2 -a o 2 3 4 INPUT T0 QUANTIZER Fig. 4

lOl

Fig.6.

VAL ADP lOl Fig.7.

VAL ADP IOI VAL ADP Fig. 9.

PATENTEUFEB 731975 SHEET USUF 2O VAL ADP VALADP s ll 2 I 3 -Z VAL ADP VALADP s 5 5 3 5 v AIADP VAL ADP s |2 2 4 6 2 VAL ADP VAL ADP e A 2D VALADP VALADP e I2 8 4 e 2 VAL ADP VAL ADP s L- ;3 2 9 -----Z3 CD VAL ADP VAL ADP 6 6 u 5 -C I 3 Z 4 v AL ADP VAL ADP s 12 a 4 6 2 6 VAL ADP VAL ADP s l3 2 u -z I I VAL ADP c 15 9 z +z 2 PAIENTEU FEB 1 3 I973 SHEET new 20 3.716.840

mnmuzanm Fi 3 SET ALL: 10= 0 saw VIACLLIEOOF N READ INPUT SIGNAL (5) AND DESIRED OUTPUT LEVEL N UNTRAINED POINT IDUM IDUM l LEVEL LEVEL +1 IDHJCFIX (LEVEL) IO(2,IC]=ID(2,IDUM) ID (2,1DUM) [C IDUM=ID (ZJDUM) I HE G) mum mum +1 YES EXECUTE: Q

mum =10 TRAIN |3A ID(2,IDUM)= ID(2, IDUMH- I IDU, IDUM)= IDU IDUMH- 2 A IDU IOUM) IID (2,1DUM' LEVEL= N ID (2,IC)= IDUM PATENTEB EB 3716.840

SHEET 07m 20 VALADP ADF N VAL ADPADF N VAL ADP G A l I 2 UN 2 3 I I 3 2 I (D Fig,

VAL ADP ADF N VAL ADP ADF N VAL ADP G A I I 2 2 1- 4 3 l 3 2 I (D L I L LVAL ADPADF N VALADP G A FI'gI/Z I2 2 5 I 4 5 2 I ED L (D VAL ADPADF N VAL ADP ADF N VAL ADP G A ll23[--H43 F-I3Z I (D L r I L LVAL ADP ADF N VAL ADP G A F/gJJ I2 2 5 2 I 4 5 2 I VAL ADPADF N VAL ADPADF N VAL ADP G A I 2 3 -l2 4 5 2 I 3 2 l L LVAL ADP ADF N AL ADP G A II 2 3 I 4 s 2 I F/ ./4 I I VAL ADP G A s s 2 I PATENIED 3.716.840

SHEET UBUF 2O VAL ADPADF N VAL ADPADF N VAL ADP e H -ll24[-@SI2452 3 L 7 I I VALADPADF N AL ADP s H F F II 7 3 I 4 6 Z2 I lg. 5 4 I L I VAL ADP ADF N VAL ADP s A l3 2 a l 5 5 2 l C VAL ADP e A s a Z vAI. ADPADF N VAL ADPADF N VAL ADP s A I I HQ 4 5 2 I 3 2| I VAL ADPADF N VAL ADP s A ll 7 3 I 54 6 2 I F/g./6 I L I VAL ADPADF N VAL ADP s A I3 9 a I s 5 2 I Q) I 5 VAL ADPADF N LVAL ADP s A -I5 I0 I 8 8 2 I VALADP s A I2 Io Z PATENTED 3973 3,716,840

SHEET UQUF 20 VAL ADP ADF N VAL ADP ADF N VAL ADP e A 2 6 I I2 4 5 2 FP- 3 Z' I Q) L I L VAL ADPADF N I VAL ADP s A 4 ll 7 3 I w 4 6 Z2 I F/g,/7 I L l VALADD ADF N VAL ADP s A l3 9 8 5 5 Z I I L VAL ADPADF N LVAL ADP e A -I5 2 I0 2 a 8 2 I vAI 'ADP s A l2 I0 2 I VALADPADF N VAL ADPADF N VAL ADP e A I|26-I245 l3z I G) L I I VALADPADF N VAL ADP s A -I5 7 I0 2 4 6 2 I fly: GD 1 L 6) VALADPADF N VAL ADT s A l3 9 8 l 5 52 I Q) I L VAL ADPADF N vAI. ADP s A II 2 3 I 8 a 2 I L VAL ADT e A PATENTEU FEB 1 SIM SHEET lUDF 2O EXECUTION KEY 2425 A=0-1 TOT=4 =4-1 TOT= 1 A=o1 TOT= -1 TOTAL=| l I TOTAL=I ALSO PATENTED FEB] 3 I975 SHEET 110F2O A: m H

muooz PATENTED FEB I 3W5 SHEET 128F 20 ENTER FROM USUAL PROCEDURE l I )VALUES OF QUANTIZERS IX (l1 IX(2),-" IX (N) (2] VALUE OF N ASSIGN WEIGHT VALUES WTU), WT(2],--WT(N) IE(I)=WT(I)* DIFUDU. IDUM] IX(I)] K (I)= [DUM READ PRE ASSIGNED VALUE FOR ITOTAL IDUM=IDUM+| ITOT ITOT- IEKI) I TOTAL INFORMATION STORAGE AT LOCATION JC ITOT ID(2, IDUM) IDUM YES IDUM K( I) OUTPUT OEUSION YES 00 WE REJECT NODE? no we ADD RESPONSE? PATENIEDFEBHIQB 3.716.840

SHEET NSF 2O REG. REG.

I64 I65 I I QUANTIZER OUANTIZER :00, mum

(IOMPARATOR IX(LEVEL) [0(2 IDUM) COMPARATOR 304 LEVEL REG.

COMPARATOR N. REG

PAIENIE FEB I 31975 INPUT SELECT INPUT I SELECT SHEET lSDF 20 Fig. 23

IC REGISTER I I I I KEY COMPONENT AND G MATR X STORAGE I ADPANO MATRIX STORAGE IDUM REGISTER OUTPUT SELECT OUTPUT SELE C T PATENTH] FEB I 31973 SWEET mmooumo PATENTEDFEBHIQYS SHEET 17GF2O PATENTED FEB I 3 I975 SHEET lQUF 2O COMPARE OUTPUT SELECT PRE-[TOTAL I TOTAL COMPARE 0 7 0 3 filo. 2 3 3 w 2 H E 

1. A method of operating a trained processor comprised of at least one memory array in which reference signals are stored along with associated trained responses forming a data base to locate and extract a desired output to query signals, which method comprises: a. in a first mode, searching through reference signals stored in a memory array comprising said at least one memory array with a query signal; b. selecting one or more of said reference signals from such memory array according to a first preselected criterion; c. retrieving from such memory array first mode trained responses associated with the reference signals selected according to said first criterion, said first mode trained responses determining the domain of a search to be conducted in a second mode; d. and in said second mode, searching through reference signals stored in a memory array comprising said at least one memory array is determined by said first mode trained responses; e. selecting one or more of said reference signals from such memory array for said query signal according to a second preselected criterion; and f. retrieving from such memory array second mode trained responses associated with the reference signals selected according to said second criterion.
 1. A method of operating a trained processor comprised of at least one memory array in which reference signals are stored along with associated trained responses forming a data base to locate and extract a desired output to query signals, which method comprises: a. in a first mode, searching through reference signals stored in a memory array comprising said at least one memory array with a query signal; b. selecting one or more of said reference signals from such memory array according to a first preselected criterion; c. retrieving from such memory array first mode trained responses associated with the reference signals selected according to said first criterion, said first mode trained responses determining the domain of a search to be conducted in a second mode; d. and in said second mode, searching through reference signals stored in a memory array comprising said at least one memory array is determined by said first mode trained responses; e. selecting one or more of said reference signals from such memory array for said query signal according to a second preselected criterion; and f. retrieving from such memory array second mode trained responses associated with the reference signals selected according to said second criterion.
 2. The method of claim 1 wherein said second mode trained responses provide the desired output of the trained processor to said query signals.
 3. The method of claim 1 wherein said at least one memory array comprises a single tree-structured array, the reference signals stored in said tree-structured array being searched in said first mode and only those reference signals selected during said first mode being searched in said second mode. 